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PREFACE 

This  Central  Research  Project  is  based  on  a  prior  IDA  study  for  the  Balanced 
Technology  Initiative  Office.  That  study  provided  not  just  the  methodology  discussed  in 
Chapter  El,  but  more  importantly  served  as  the  foundation  for  the  analytic  techniques 
described  in  this  document.  The  members  of  that  study  team  included:  Dr.  Richard  E. 
Schwartz,  project  leader.  Dr.  Peter  S.  Brooks,  Dr.  Frederic  A.  Miercort,  Dr.  David 
Spalding,  and  Ms.  Marchelle  M.  Stahl. 

The  authors  thank  the  reviewers  for  their  assistance-Dr.  Peter  S.  Brooks  and 
Dr.  William  J.  Hurley-and  also  Dr.  Jesse  Orlansky  for  his  comments.  Ms.  Paula  B. 
Greer  provided  editorial  assistance. 
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I.  INTRODUCTION 

This  paper  describes  a  general  analytic  methodology  for  processing  and 
manipulating  distributed  interactive  simulation  (DIS)  data  derived  from  one  or  more 
exercises.  This  is  introduced  by  a  case  study  describing  how  simulation  network 
(SIMNET)  data  were  used  in  support  of  a  specific  analysis.  The  case  study  both 
illustrates  the  general  methodology  and  serves  as  an  example  of  how  SIMNET  (as  well  as 
DIS)  data,  operational  data,  and  modeling  results  can  be  combined  to  produce  meaningful 
analytic  results. 

A  note  on  nomenclature:  DIS  has  multiple  definitions.  The  narrowest  use  of  DIS 
refers  to  exercises  that  use  the  DIS  1.0  or  2.x  protocols.  These  protocols  define  the 
format  and  structure  of  communications  among  various  simulations  or  manned 
simulators.  DIS  is  also  used  in  a  broader  sense  to  describe  any  simulation  that  is 
distributed  and  interactive,  e.g.,  Janus. 

In  this  paper,  DIS  will  be  used  in  the  narrowest  sense.  The  predecessor  to  DIS 
was  the  SIMNET  protocol  [also  the  name  of  the  Defense  Advanced  Research  Projects 
Agency  (DARPA)  program  that  sponsored  its  development].  SIMNET  is  used  when 
referring  to  specific  exercises  conducted  using  the  SIMNET  protocol. 

The  paper  begins  with  an  abbreviated  description  of  DIS,  highlighting  only  those 
aspects  relevant  to  this  paper.  The  second  section  describes  the  use  of  SIMNET  data  to 
evaluate  the  potential  operational  value  of  two  proposed  weapon  systems. 

The  third  section  describes  an  analytic  methodology  that  focuses  on  exploiting 
DIS  data  from  previously  recorded  exercises  to  produce  results  for  issues  other  than  those 
for  which  the  exercise  was  originally  planned.  This  approach  assumes  that,  for  some 
systems  and  for  some  issues  (examples  are  discussed  later  in  the  section),  instead  of 
planning  and  executing  a  computer  simulation  analysis  or  a  live  DIS  experiment,  one  can 
develop  interesting  insights  and  results  from  archived  SIMNET  or  DIS  exercises 
originally  conducted  for  other  purposes,  e.g.,  training.  Many  SIMNET  and  DIS  exercises 
have  been  archived  and  are  available  for  this  use,  representing  a  huge  database  of  manned 
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battles.  In  addition,  if  a  full-scale  live  experiment  is  conducted,  the  techniques  described 
in  this  paper  can  be  used  to  extend  the  results. 

Three  techniques  for  processing  and  analyzing  DIS  data  are  discussed.  The  first 
technique,  basic  processing  of  DIS  data,  is  used  to  develop  measures  of  tactical  or 
from  one  or  more  DIS  exercises.  The  second  technique  involves 
imposing  a  model  on  a  DIS  exercise,  thereby  introducing  a  new  system  into  the  battle 
after  it  has  occurred.  In  the  third  technique,  a  DIS  exercise  is  replayed  to  a  crew  in  a 
manned  simulator  that  can  then  participate  in  the  previously  recorded  battle.  These 
techniques  and  their  associated  limitations  and  advantages  are  discussed  in  detail. 
Illustrative  applications  of  each  are  presented.  As  large-scale  manned  distributed 
simulation  technology  matures,  the  applications  and  importance  of  these  techniques  will 
grow. 
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II.  BACKGROUND 

SIMNET  was  initiated  by  DARPA  in  1983  to  develop  and  demonstrate-through  a 
synthesis  of  computer,  computer  networking,  and  display  technologies-manned 
simulators  for  tactical  simulation  and  training.  Unlike  highly  realistic,  complex,  and 
expensive  manned  simulators  that  are  used  to  train  individuals  or  crews  in  the  operation 
of  a  vehicle,  SIMNET  was  designed  to  train  crews  (that  already  know  how  to  operate  a 
vehicle)  to  fight  a  battle.  Each  simulator  is  relatively  inexpensive.  SIMNET  emphasizes 
the  interactions  among  the  many  vehicles  present  on  a  battlefield,  so  that  many  manned 
simulators  are  needed  to  conduct  a  simulated  battle.  The  information  (primarily  visual 
images)  that  would  be  available  to  a  crew  on  a  battlefield  and  the  controls  (primarily 
movement  and  weapons  controls)  available  to  a  crew  to  fight  a  battle  are  simulated. 

The  SIMNET  program  ended  in  1990,  and  responsibility  for  the  program  and  the  - 
simulators  developed  by  DARPA  to  date  was  passed  to  the  Army.  DARPA  and  the 
Services,  primarily  the  Army,  have  continued  to  invest  in  the  technology  and  an  updated 
protocol,  DIS,  has  been  introduced  and  continues  to  evolve.  DIS  simulators  are 
conceptually  similar  to  their  SIMNET  counterparts  but  differ  in  some  important  aspects. 
In  addition  to  using  the  newer  protocol,  they  tend  to  be  of  higher  fidelity  and  often  have 
more  sophisticated  image  generMors  that  create  better  visual  effects.  SIMNET  simulators 
are  still  in  use,  principally  in  the  training  community. 

A.  OPERATIONAL  CONFIGURATION  &  CAPABILITIES 

In  DIS,  computer-generated  entities  (vehicles  or  weapons)  under  human  control 
interact  in  real  time.  Battlefield  entities  are  typically  created  and  controlled  by  manned 
simulators,  although  other  programs  can  accomplish  that  also.  The  various  programs  and 
simulators  controlling  entities  communicate  with  each  other  by  sending  data  in  a  specific 
format  over  a  common  network. 

As  an  example  of  a  DIS  battle,  suppose  a  trial  is  to  be  made  with  a  defensive  force 
of  5  tanks  and  an  attacking  force  of  15  tanks.  The  five  tank  simulators  may  be  located  at 
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Fort  Knox  and  the  offensive  force  in  Germany.  However,  the  terrain  over  which  the 
battle  occurs  might  be  an  Army  training  range  in  California. 

Each  simulator  in  this  example  represents  a  tank.  Current  tank  simulators 
represent  the  inside  of  the  Ml,  but  they  may  be  given  different  characteristics  by 
modifying  their  software.  For  example,  they  might  have  different  mobility  or  fire  control 
characteristics,  weapon  effectiveness,  etc.  The  unit  commander  on  each  side  is  able  to 
communicate  with  his  subordinates  by  radio.  The  fidelity  of  the  simulator  (i.e.,  how 
realistic  the  workstations  are  and  the  degree  to  which  combat  effects  such  as  movement, 
vibration,  and  sound  are  simulated)  is  a  function  of  investment.  Some  may  be  much  more 
realistic  than  others  in  the  same  network. 

As  vehicles  move,  each  broadcasts  its  current  position  and  status  to  the  others  in 
the  network.  Line  of  sight  (LOS)  for  the  selected  terrain  and  range  determine  when  each 
vehicle  appears  on  the  display  screen  of  other  simulators.  The  picture  displayed  is 
controlled  by  the  software,  so  different  representations  may  be  used  for  friendly  and 
enemy  units.  When  the  crew  detects  and  identifies  an  enemy  unit,  and  decides  to  fire, 
each  crew  member  goes  through  the  firing  sequence  for  his  station.  The  simulators  have 
a  full  crew—a  commander,  driver,  gunner,  and  loader— although  some  of  these  could  be 
simulated  rather  than  actual.  At  the  appropriate  command,  a  round  is  fired.  This 
information  is  passed  to  all  other  vehicles  so  that  the  software  may  decide  whether  to 
display  the  firing  signature.  The  computer  of  the  firing  unit  then  calculates  the 
probability  of  hit  depending  on  the  range  and  exposure  of  the  target,  the  type  of  round  and 
target,  and  the  accuracy  of  the  fire  control  solution  and  weapon.  If  the  target  is  hit,  this 
information  is  passed  to  the  computer  of  the  target.  A  representation  of  the  hit  is  shown 
on  the  graphics  of  all  vehicles  that  can  see  it.  The  unit  that  is  hit  calculates  the  extent  of 
the  damage. 

In  the  same  trial  one  of  the  simulators  could  be  a  command  post  (CP).  The  CP 
would  receive  information  from  its  units  in  the  field  as  they  detect  the  enemy  or  as  their 
status  changes.  For  example,  they  may  need  fuel,  ammo,  or  maintenance.  They  can 
report  this  and  the  CP  unit  schedules  the  action,  or  the  unit  may  request  fire  support,  with 
new  simulators  entering  the  game  to  provide  artillery  fire  or  close  air  support. 
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B.  MANNED  SIMULATORS 

Manned  simulators  are  physical  mock-ups  of  the  interior  of  the  vehicle  they  are 
intended  to  represent.  The  simulators  include  physical  representations  of  crew  quarters, 
including  controls  needed  for  vehicle  movement  and  weapons  systems.  A  sound  system 
provides  battlefield  and  vehicle  machinery  noise,  and  a  visual  display  system  shows  a 
battlefield  image  to  the  crew.  Vehicle  or  turret  movement  is  simulated  by  changes  in  the 
visual  presentation  to  the  crew. 

Each  simulator  includes  computer  equipment  that  stores  and  processes  a  digitized 
terrain  database.  The  display  system  shows  this  terrain  and  superimposes  on  it  images  of 
moving  tanks,  missiles,  explosions,  etc.,  as.  needed  to  simulate  the  dynamic  battle.  Each 
simulator  generates  its  own  imagery  and  very  short  messages  [called  protocol  data  units 
(PDUs)]  to  communicate  its  position  and  status  to  the  other  simulators.  Processing 
digital  simulated  imagery  in  real  time  requires  substantial  computational  power,  and  the 
display  systems  are  the  most  sophisticated  parts  of  the  simulators.  The  tank  simulators 
have  displays  for  the  different  vision  blocks  of  the  tank  but  not  for  open  hatch  views.  A 
tank’s  visual  range  is  limited  to  about  3.5  kilometers  (km),  and  the  terrain  polygons  are 
generated  typically  from  elevation  data  for  points  separated  by  125  meters.  These 
limitations  are  not  inherent  but  represent  practical  cost  and  technology  constraints. 

Digitized  terrain  is  typically  derived  from  Defense  Mapping  Agency  data.  Most 
terrain  databases  are  rectangular,  covering  about  50  to  100  km  on  a  side.  Each  vehicle 
controls  its  own  (simulated)  movement  over  the  terrain.  Factors  such  as  fuel  and  terrain 
slope  are  considered.  Based  on  the  messages  it  receives  during  the  battle,  each  simulator 
presents  appropriate  visual  displays  and  determines  the  damage  it  sustains  if  hit. 

C.  COMPUTER-GENERATED  FORCES 

In  addition  to  manned  simulators,  battlefield  entities  may  be  controlled  by 
computer-generated  forces  (CGF).  CGF  programs  allow  an  operator  to  control  various 
echelons  (e.g.,  companies  or  battalions)  of  simulated  vehicles.  These  vehicles  are 
represented  entirely  by  software  and  not  by  manned  simulators.  The  CGF 
operator/commander  provides  higher  level  commands,  e.g.,  routes  to  follow,  and  the 
software  controls  the  detailed  operation  of  the  vehicles  (including  motion  and  weapons 
fire).  CGF  interact  with  other  DIS  elements  in  the  same  way  as  manned  simulators.  CGF 
cannot  be  distinguished  from  manned  simulator  vehicles  by  their  appearance,  and  the  goal 
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is  to  make  them  indistinguishable  on  the  basis  of  behavior;  this  goal  has  not  been  fully 
achieved. 

D.  DATA  COLLECTION 

PDUs  broadcast  over  the  network  by  manned  simulators,  CGF,  or  other  programs 
are  captured  and  recorded  by  a  program  called  a  logger.  This  program  tags  each  PDU 
with  a  time  stamp  and  writes  the  data  to  either  a  tape  or  a  disk.  These  data  files  constitute 
a  complete  record  of  the  observable  interactions  that  occur  during  a  DIS  exercise.  Data 
on  internal  activities  such  as  weapons  aiming  are  not  collected.  The  exercise  can  be 
reenacted  by  “playing  back”  these  data  records  over  a  network.  Such  playbacks  are 
indistinguishable  from  real-time  observations  of  the  actual  exercise  by  nonparticipants. 

In  order  to  analyze  DIS  data,  they  must  be  extracted  from  the  data  files  created  by 
the  logger.  Identifying  particular  PDU  types  is  straightforward,  so  that  some  information 
(e.g.,  weapons  fired,  tanks  killed)  is  easily  obtained.  Other  information,  such  as 
intervisibility,  is  not  directly  recorded  and  must  be  determined. 

In  addition  to  logger  files,  generally  one  or  more  of  the  following  types  of  data  are 
collected  during  an  exercise.  These  data  are  less  amenable  to  computer-based  analysis 
than  logger  files  but  they  may  contain  important  information.  They  are  mentioned  here 
for  completeness  but  are  not  discussed  further  in  this  paper. 

•  Voice  recordings  of  radio  traffic-Typically  groups  of  simulators  are  linked  by 
a  CB  radio,  and  radio  traffic  over  one  or  more  channels  is  often  recorded 
during  an  exercise. 

•  Video  recordings  of  operators— In  some  exercises,  cameras  in  the  simulators 
record  operator  actions  to  provide  data  for  human  factors  evaluation. 

•  Data  collected  by  observers— Observers  may  be  stationed  in  simulators, 
sometimes  doubling  as  crew  members,  to  record  actions  of  the  operators. 

•  Interviews  with  participants— Interviews  may  be  conducted  before  and  after  an 
exercise.  These  provide  information  about  the  backgrounds  and  capabilities 
of  the  participants,  their  opinions  on  the  realism  of  the  simulation,  and 
simulator  ergonomics. 
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E.  STRENGTHS  AND  WEAKNESSES  OF  MANNED  DIS 

The  strength  of  manned  DIS  exercises  is  the  incorporation  of  human  interactions 
and  decision-making  in  simulated  combat.  In  DIS  as  in  real  life,  soldiers  have  imperfect 
information  on  which  to  base  their  decisions:  vehicle  positions  are  estimated, 
identification  friend  or  foe  (IFF)  is  inaccurate,  etc.  Decision-making  takes  place  at  two 
levels.  The  lower  level  involves  the  interaction  among  the  crew  and  between  the  crew 
and  a  simulator.  Software  defines  the  capabilities  of  the  vehicle  simulated  and  the  way  in 
which  it  will  respond  to  operator  commands.  The  operator  decides  how  to  use  the 
vehicle/simulator:  how  to  maneuver,  where  to  stop,  when  to  fire,  what  to  fire  at.  At  a 
more  aggregate  level,  commanders  at  the  same  or  different  levels  communicate  with  each 
other  over  radio  networks.  Tactical  responsibility  is  assigned  at  various  levels:  vehicle 
commander,  platoon,  and  company  commander,  etc. 

DIS  can  be  used  to  address  broad  issues,  such  as  the  effect  that  an  innovative 
weapon,  munitions,  or  set  of  tactics  might  have  on  the  battle.  Examples  of  this  are  the 
Counter  Target  Acquisition  System  test,  which  evaluated  the  opportunity  for  an  enemy 
laser  system  to  attack  pilots’  vision,  and  the  M1A2  Block  n  tests  that  measured  the  effect 
of  various  components  (a  commander’s  thermal  viewer,  an  accurate  positioning  system, 
and  an  inter- vehicle  position  communication  system)  on  the  performance  of  the  M1A2. 
Command  and  control  issues  can  also  be  evaluated. 

DIS  exercises  produce  data  about  weapon  or  munition  performance,  such  as  the 
probability  of  killing  a  particular  target  type  or  the  number  of  rounds  required  to  destroy  a 
particular  target.  However,  certain  performance  parameters,  e.g.,  the  probability  of  kill 
given  a  hit  for  a  particular  round,  are  inputs  to  a  DIS  exercise  and  therefore  cannot  be 
outputs.  The  point  to  note  is  that  using  DIS  does  not  allow  for  estimation  of  performance 
parameters  of  individual  systems  that  are  independent  of  their  operators;  rather,  DIS  can 
be  used  to  measure  the  effect  of  those  parameters  under  the  conditions  of  the  exercise 
including  human  control. 

In  addition,  DIS  exercises  provide  data  for  various  human  factors  issues.  Some 
issues  are  in  the  category  of  operator  workload  and  performance:  how  long  it  takes  an 
operator  to  complete  a  certain  task  or  how  accurately  the  task  is  performed.  Other  issues 
relate  to  simulator  ergonomics  and  the  effectiveness  of  the  man-machine  interface. 
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DIS  suffers  from  the  following  limitations,  which  make  it  difficult  to  use  for 
analytic  experiments: 

•  Low  Fidelity  Graphics-The  cost  of  a  computer  image  generator  is  an 
important  factor  in  overall  simulator  cost.  Typically,  lower  cost  generators  are 
used  and  the  resulting  images  are  somewhat  lacking  in  realism.  This  makes 
results  on  operator  detection  and  identification  of  targets  highly  suspect.  (This 
may  be  offset  by  combining  results  from  DIS  exercises  with  results  from  field 
tests  as  described  in  the  next  section.  Alternatively,  if  one  can  establish  that 
detection  and  identification  in  DIS  is  easier  than  in  real  life,  then  it  may  be 
useful  to  view  DIS  results  as  an  upper  bound  on  potential  operational  results.) 

•  Low  Fidelity  Models-Simulators  are  designed  for  a  variety  of  purposes  and 
the  underlying  system  models  may  not  support  the  fidelity  needed  for  a  given 
analytic  experiment. 

•  Terrain-Most  terrain  databases  represent  terrain  by  flat  adjacent  125-meter  x 
125-meter  polygons,  which  meet  in  sharp  edges.  More  detailed  terrain  is 
desired  for  specific  applications,  such  as  evaluating  various  terrain  traversal 
algorithms  for  unmanned  ground  vehicles  (UGVs).  Dynamic  terrain,  which 
changes  as  the  battle  unfolds,  e.g.,  becoming  pocked  from  munitions'  impacts 
on  the  ground,  is  just  starting  to  be  used  but  is  not  widespread. 

•  Inadequate  Data  Collection  for  Analysis-The  DIS  and  SIMNET  protocols’ 
main  focus  is  to  support  the  correct  visual  displays  on  a  simulator.  Detailed 
information  about  internal  system  behaviour  is  not  part  of  the  protocol  and  is 
generally  not  available.  For  many  experiments,  simulators  are  modified  to 
produce  additional  data  beyond  that  required  by  protocol. 


UNCLASSIFIED 


UNCLASSIFIED 


III.  CASE  STUDY:  EVALUATION  OF  FIBER 
OPTIC  GUIDED  (FOG)  WEAPON  CONCEPTS 

A.  BACKGROUND 

In  1991,  IDA  undertook  a  study  for  the  Balanced  Technology  Initiative  Office  to 
assess  the  potential  operational  value  of  two  FOG  weapon  concepts:  mortar  rounds  in 
two  calibers--81  mm  [denoted  FOG-MP(81)]  and  120  mm  [denoted  FOG-MP(120)]— and 
a  helicopter-launched  missile  (denoted  FOG-HM). 

FOG  munitions  have  three  components:  a  projectile  with  an  imaging  sensor,  a 
control  station,  and  a  fiber  optic  cable.  As  the  FOG  projectile  flies  out,  the  fiber  optic 
cable  is  payed  out  behind  it.  During  fly-out,  images  of  the  battlefield  are  transmitted 
from  the  projectile's  sensor  to  the  control  station  where  they  are  displayed  to  an  operator. 
The  operator  searches  the  scene  for  targets,  performs  IFF,  selects  a  target,  and  locks  the 
projectile  onto  the  target.  The  operator  enters  appropriate  commands  for  these  actions 
into  the  control  console,  which  then  transmits  control  signals  through  the  fiber  optic  cable 
to  the  FOG  projectile. 

The  dependence  on  the  human  operator  to  detect  and  select  targets  is  both  the 
principal  advantage  and  the  principal  disadvantage  of  FOG  systems.  Benefits  may 
inelude:  last-minute  IFF  to  minimize  friendly  losses,  visual  target  detection  and  aimpoint 
selection  to  destroy  specific  targets  with  minimal  collateral  damage,  and  the  opportunity 
to  gain  incidental  intelligence  during  projectile  fly-out.  The  disadvantage  is  that  the 
gunner’s  involvement  limits  the  rate  of  target  engagement. 

Because  the  gunner  plays  a  major  role  in  the  performance  of  fiber  optic  systems, 
the  analytic  approach  was  oriented  toward  the  use  of  data  from  tests  of  two  Army  ground- 
launched  FOG  systems;  the  FOG  missile  (FOG-M)  and  its  successor,  the  Non-Line-of- 
Sight  (NLOS)  missile: 

•  An  initial  operational  evaluation  (lOE)  of  the  FOG-M  was  conducted  at 
Redstone  Arsenal,  Alabama,  and  White  Sands  Missile  Range,  New  Mexico, 
between  September  1988  and  September  1989.  The  lOE  included  captive 
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carry  flight  tests  of  the  seeker  designed  to  evaluate  the  gunner’s  ability  to 
detect  a  target  presented  in  the  sensor’s  field  of  view  (FOV).  The  sensor  was 
mounted  on  the  underside  of  an  airplane  and  flown  in  a  way  that  would  bring 
targets  (both  tanks  and  hovering  helicopters)  into  the  sensor  FOV.  The  sensor 
imagery  was  displayed  to  the  gunner  who  attempted  to  detect  the  target  and 
lock  on  to  it.  The  lOE  also  included  live  missile  firings  to  evaluate  the 
missile’s  ability  to  hit  a  target  after  lock-on.  The  lOE  has  been  described  and 
analyzed  by  Richardson  et  al  (1989). 

•  The  NLOS/SIMNET  evaluation,  using  manned  NLOS  simulators,  was 
conducted  in  April  1991  at  the  Fort  Rucker,  Alabama,  SIMNET  site  by  the 
Army  Operational  Test  and  Evaluation  Command  (OPTEC).  One  purpose  of 
the  evaluation  was  to  determine  the  effect  of  different  conditions  on  target 
availability  for  the  NLOS  missile.  The  primary  question  was,  given  different 
time  delays  between  target  acquisition  by  a  sensor  or  forward  observer  and  the 
cueing  of  the  NLOS  gunner  to  that  target,  would  the  target  still  be  available 
for  detection  by  the  time  the  missile  arrived  at  the  cued  target  location? 
OPTEC  published  the  results  in  1992. 

•  Because  the  major  portion  of  the  analysis,  described  later  in  this  section,  was 
based  on  the  NLOS/SIMNET  data,  some  background  on  the  NLOS/SIMNET 
evaluation  is  necessary. 

B.  THE  NLOS/SIMNET  EVALUATION 

One  purpose  of  the  NLOS/SIMNET  evaluation  was  to  determine  the  effect  of 
various  conditions,  primarily  cue  delay,  on  target  availability  for  the  NLOS  missile.  The 
cue  delay  is  the  time  interval  between  the  acquisition  of  the  target  by  a  sensor  or  forward 
observer  and  receipt  of  a  target  cue  by  the  NLOS  gunner.  The  issue  is  whether  a  given 
cue  delay  allows  sufficient  time  for  the  NLOS  missile  to  get  to  the  target  area  and  find  the 
target.  A  target  is  available  for  detection  if  one  or  more  target  vehicles  is  within  the 
sensor  FOV,  within  the  maximum  sensor  detection  range,  and  within  LOS  of  the 
projectile,  irrespective  of  whether  or  not  the  gunner  actually  detects  and  engages  the 
target. 

Two  identical  NLOS  simulators  were  developed  for  the  evaluation.  Figure  1 
depicts  an  NLOS  simulator.  The  simulator  represents  the  cab  of  a  High  Mobility 
Multipurpose  Wheeled  Vehicle  (HMMWV);  the  NLOS  missile  is  intended  to  be  launched 
from  the  rear  of  the  vehicle.  In  front  of  the  cab  is  a  large  curved  screen  that  displays  the 
SIMNET  battlefield  as  it  would  appear  looking  through  the  HMMWV’ s  windshield. 
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Eight  hundred  forty  (840)  single  shot  trials  were  run  to  provide  data  for  analysis. 
Each  trial  began  with  the  HMMWV  emplaced  on  the  battlefield;  the  NLOS  missile 
cannot  be  fired  while  the  vehicle  is  in  motion.  The  gunner  was  seated  in  the  simulator, 
awaiting  a  target  cue.  The  cue,  coming  over  a  radio  or  through  the  NLOS  computer 
located  in  front  of  the  gunner,  would  describe  the  type  of  target,  armor  or  rotary  wing,  and 
the  location  of  the  initial  target  sighting.  (Although  4  to  190  seconds  had  passed  since 
that  sighting,  the  gunner  was  not  told  the  age  of  the  information).  The  gunner  would 
enter  information  into  his  NLOS  computer  and  fire  the  missile.  After  a  20-second  delay, 
the  missile  would  be  launched. 

During  missile  fly-out,  the  gunner  watched  the  small  screen  in  front  of  him.  This 
displayed  the  view  of  the  SIMNET  battlefield  as  seen  from  the  simulated  TV  sensor 
located  in  the  nose  of  the  missile.  The  missile  is  programmed  with  a  default  flight  path  to 
the  cued  target  location  and  with  a  default  sensor  sweep  pattern,  so  all  that  was  required 
of  the  gunner  was  to  watch  the  screen,  detect  a  target,  and  lock  on  to  the  target.  However, 
the  gunner  could  take  manual  control  of  the  missile  or  sensor  using  the  joystick  located 
on  the  armrest,  and  in  most  trials,  he  did.  (One  purpose  of  the  analysis,  described  below, 
was  to  determine  the  effect  of  manual  seeker/missile  control  on  target  availability.)  Each 
trial  ended  when  the  missile  hit  either  the  ground  or  a  target. 

OPTEC  provided  the  NLOS/SIMNET  test  plan  and  copies  of  the  21  DataLogger 
tapes  used  to  record  the  840  trials  and  patiently  answered  many  questions  about  the  test. 
Data  on  705  of  the  trials  were  usable  but  data  on  the  remaining  135  trials  were  flawed, 
primarily  due  to  errors  in  the  simulators. 
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C.  OVERVIEW  OF  THE  ANALYTIC  APPROACH 

The  engagement  performance  of  the  FOG  mortar  round  and  the  FOG-HM  were 
defined  to  be  the  probability  of  successful  single  shot  engagement  [P(SSE)].  This  is  the 
product  of  the  following  terms: 

•  The  probability  that  a  target  is  available  for  detection 

•  The  probability  that  the  target  is  detected  and  locked  onto  given  that  it  is 
available 

•  The  probability  that  the  target  is  hit  given  that  it  is  locked  onto 

•  The  probability  that  the  target  is  killed  given  that  it  is  hit. 

At  the  outset  of  the  study,  suitable  estimates  were  available  or  derivable  for  the 
last  three  terms  but  not  for  the  probability  of  target  availability.  Therefore,  developing  a 
P(SSE)  for  each  FOG  concept  required  developing  an  estimate  of  target  availability. 
These  estimates  were  based  on  data  from  the  NLOS/SIMNET  evaluation. 

Figure  2  is  an  overview  of  the  analysis.  There  are  four  parts  to  the  analysis: 

•  Determining  target  availability  for  the  NLOS  missile  in  the  SIMNET 
evaluation  and  the  effect  of  different  conditions  on  target  availability 

•  Determining  the  effect  of  operator  control  on  NLOS  target  availability 

•  Developing  an  estimate  of  target  availability  for  the  FOG  concepts,  based  on 
NLOS/SIMNET  data 

•  Combining  the  estimate  of  target  availability  with  estimates  of  the  three  other 
terms  to  create  a  P(SSE)  for  each  of  the  FOG  concepts. 

In  describing  the  analysis,  emphasis  is  placed  on  the  methodology;  some 
illustrative  results  are  presented. 

1.  NLOS  Target  Availability 

Referring  to  box  1  in  Figure  2,  the  first  step  in  the  analysis  was  to  process  the 
NLOS/SIMNET  data  and  determine  target  availability.  General  purpose  processing 
software  was  developed  to  read  the  DataLogger  tapes,  determine  the  positions  of  all 
vehicles  and  missiles  for  each  second  of  each  trial,  compute  target  availability  for  each 
trial,  and  produce  other  processed  data  for  each  trial. 

Target  availability  is  computed  at  each  second  in  a  trial  in  the  following  way. 
First,  from  the  SIMNET  data  on  a  DataLogger  tape,  the  program  determines  the  location 
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of  all  the  vehicles  in  the  trial  and  the  location  of  the  NLOS  missile  relative  to  the 
digitized  terrain.  (A  digitized  version  of  Fort  Hunter-Liggett,  California,  was  used  in  the 
NLOS/SIMNET  evaluation  and  therefore  in  the  availability  calculations.)  It  calculates 
the  missile’s  sensor  FOV,  projects  it  onto  the  terrain,  and  determines  whether  any 
vehicles  are  within  the  sensor  FOV.  If  so,  it  then  calculates  whether  intervisibility  exists 
between  the  sensor  and  each  vehicle  in  the  FOV.  It  also  computes  whether  the  range  to 
each  vehicle  in  the  FOV  is  less  than  the  sensor’s  (specified)  maximum  detection  range. 

The  availability  program  produces  a  variety  of  information  about  each  trial.  From 
these  data,  the  705  trial  outcomes  were  compiled  in  various  ways  to  indicate  the  factors 
that  were  important  in  determining  target  availability. 

On  average,  targets  were  available  for  detection  in  46  percent  of  the  trials. 


Figure  2.  Analysis  Overview 
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Table  1  shows  the  influence  of  cue  delay,  cue  type,  and  target  type  on  target 
availability.  Target  availability  is  high  with  a  cue  delay  of  less  than  40  seconds,  and  low 
with  cue  delays  of  40  seconds  or  more.  Table  2  describes  the  method  of  target  acquisition 
that  was  represented  by  each  cue  delay.  The  NLOS  missile  has  high  target  availability 
when  cues  are  either  provided  in  the  auto  mode  or  by  a  forward  observer  calling  targets 
directly  to  the  NLOS  unit.  In  addition  to  using  those  target  acquisition  methods  listed  in 
Table  2  with  short  cue  delays,  other  ways  to  improve  target  availability  would  be  to 
significantly  decrease  the  delay  between  giving  the  command  (electro-mechanically)  for 
the  missile  to  fire  and  the  actual  launch  of  the  missile,  or  to  provide  the  NLOS  with  a 
dedicated  target  acquisition  capability,  such  as  an  unmanned  air  vehicle  (UAV). 


Table  1.  (U)  Fraction  of  Trials  in  Which  NLOS  Target  Was  Available  By  Target 

Type,  Cue  Type,  Cue  Delay 


Target 

Cue 

Cue  Delay  (sec) 

Type 

Type 

4 

15 

40 

85 

140 

190  1 

Rotary 

Auto 

53/58 

(0.91) 

44/59 

(0.75) 

30/60 

(0.50) 

IB 

Manual 

24/30 

(0.80) 

24/60 

(0.40) 

118/294 

(0.40) 

4/59 

(0.07) 

2/30 

(0.07) 

Armor 

Manual 

12/25 

(0.48) 

11/30 

(0.37) 

Table  2.  (U)  Cueing  Data  Source  by  Mode  and  Delay 


Cue  Mode 

Cue  Delay 
(sec) 

Data  Source 

4 

Fonward  Area  Air  Defense  System  Command  Control 
and  Intelligence  sensor 

Auto 

15 

Non-Cooperative  Target  Recognition  sensor 

40 

Included  for  analytic  purposes 

15 

Forward  observer-quick  fire  cue 

40 

Line-of-Sight-Forward-Heavy  sensor  or  element  within  a 
task  force 

Manual 

85 

Manual  SHORAD  Control  System 

140 

Included  for  analytic  purposes 

190 

Forward  observer 
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An  unanticipated  and  interesting  result  from  this  portion  of  the  analysis  is  the 
effect  of  gunner  background  on  target  availability.  Table  3  shows,  for  each  of  eight 
gunners  used  in  the  evaluation,  the  number  of  trials  in  which  the  gunner  had  a  target 
available  for  detection  divided  by  the  total  number  of  trials  in  which  the  gunner 
participated,  and  the  gunner’s  background.  Gunner  performance  varied  widely.  The  best 
gunner  found  targets  available  for  detection  in  61  percent  of  his  trials,  while  the  worst 
gunner  found  targets  available  for  detection  in  34  percent  of  his  trials.  Based  on 
interviews  with  the  gunners,  it  was  found  that  the  two  best  gunners  had  the  longest 
experience  with  their  NLOS  platoons,  had  previous  field  test  experience  with  the  FOG-M 
(the  prototype  version  of  the  NLOS  missile),  and  had  previous  computer  experience. 
This  suggests  that  gunner  training  will  be  an  important  factor  influencing  the 
effectiveness  of  FOG  weapons,  should  such  a  system  be  deployed. 


Table  3.  (U)  NLOS  Target  Availability  by  Gunner  ID  and  Background 


Gunner  ID 

Successful 

Trials/Total 

Trials 

Time  With 
NLOS  Unit 
(Months) 

NLOS 

Test 

Experience 

Computer 

Background 

(Y/N) 

6 

61/100 

(0.61) 

36 

Initial 

Operational 

Evaluation 

Y 

3 

13 

Field  Training 
Exercise 

Y 

8 

41/83 

(0.49) 

6 

None 

N 

7 

40/84 

(0.48) 

6 

None 

Y 

4 

38/96 

(0.40) 

5 

None 

N 

5 

39/102 
(0.38)  . 

3 

None 

N 

1 

25/70 

(0.36) 

11 

None 

N 

2 

27/79 

(0.34) 

10 

None 

N 
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2.  Effect  of  Operator  Control  on  NLOS  Target  Availability 

The  second  step  in  the  analysis,  labeled  box  2  in  Figure  2,  was  to  determine  the 
effect  of  operator  control  on  target  availability.  This  was  done  by  comparing  target 
availability  outcomes  of  the  705  manned  NLOS/SIMNET  trials  to  target  availability 
outcomes  for  the  same  705  trials  where  the  actual  NLOS  seeker  orientation  and  missile 
path  were  replaced  with  their  default  behavior. 

Consistent  with  the  actual  NLOS  missile,  in  the  SIMNET  trials,  the  missile  was 
programmed  to  fly  out  to  the  cued  target  location  without  any  intervention  required  by  the 
operator.  The  missile  would  behave  according  to  the  missile  model  contained  within  the 
simulator  software.  The  default  behavior  of  the  seeker  was  also  programmed  into  the 
simulator  software.  However,  if  the  operator  chose  to  do  so,  he  could  fly  the  missile 
manually  and/or  move  the  seeker  manually.  This  terminated  the  default  operation  of  the 
missile  or  seeker.  Thus,  in  each  SIMNET  trial,  any  variation  from  the  default  operation 
of  the  missile  and  seeker  is  due  to  the  operator's  taking  control. 

To  establish  what  each  trial's  outcome  would  have  been  if  the  missile  and  seeker 
always  followed  the  default  behavior,  a  deterministic  model  of  the  default  NLOS  missile 
fly-out  and  seeker  behavior  was  developed  based  on  documentation  of  the  NLOS 
simulator  software.  This  model  was  run  for  each  of  the  705  NLOS/SIMNET  trials,  using 
the  same  initial  conditions  that  had  been  used  for  each  SIMNET  trial.  The  missile  was 
launched  at  the  same  time  by  the  model  as  it  had  been  launched  in  the  SIMNET  trial,  and 
it  was  flown  to  the  cued  target  location  for  that  trial.  For  each  trial,  the  output  of  the 
model  was  the  default  missile  flight  path  and  the  default  seeker  orientation  for  that  trial. 

Target  availability  was  then  computed  for  this  hypothetical  trial  by  using  the 
model-produced  fly-out  path  and  seeker  orientation  along  with  the  SIMNET  data  on 
target  paths  and  the  Fort  Hunter-Liggett  digitized  terrain  as  inputs  to  the  availability 
program.  This  program  performed  the  FOV,  LOS,  and  range  calculations  using  the 
altered  missile  location  and  sensor  orientation  with  the  actual  target  paths  for  a  trial. 

The  effect  of  the  operator  on  target  availability  is  measured  by  the  difference  in 
outcome  between  the  705  SIMNET  trials  where  the  operator  could  take  control  of  the 
missile  and  seeker,  and  the  705  trials  where  the  missile  and  seeker  performed  in  their 
default  fashion.  Average  target  availability  is  0.46  for  the  SIMNET  trials  and  0.28  for  the 
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modeled  trials.  Thus  the  average  effect  of  human  operation  in  the  NLOS/SIMNET 
evaluation  is  to  increase  availability  by  0.18  (i.e.,  0.46-0.28). 

Operators  improved  target  availability  in  two  ways.  First,  they  searched  for 
targets  that  had  moved  away  from  the  cued  target  location  by  the  time  the  missile  arrived 
there.  This  is  not  part  of  the  default  missile  behavior.  Second,  in  some  trials  where  the 
gunners  were  given  additional  information  about  the  target,  e.g.,  in  which  direction  it  had 
been  moving  when  it  was  first  acquired,  its  speed,  and  the  time  since  it  was  acquired,  they 
could  sometimes  use  that  information  to  successfully  predict  where  the  missile  could 
intercept  the  target.  A  third  way  operators  may  have  improved  target  availability  was  by 
developing  preplanned  missile  flight  paths  to  anticipated  target  areas  during  mission 
planning.  If  target  cues  were  for  these  areas  and  the  preplanned  path  was  used,  it  may 
have  improved  target  availability  by  flying  the  missile  along  more  likely  target  paths.  No 
data  were  recorded  to  gauge  this. 

Although  the  overall  effect  of  the  operators  was  to  increase  target  availability, 
they  decreased  it  slightly  in  trials  where  the  target  was  approximately  stationary.  In  these 
trials,  the  model  had  slightly  better  availability  results,  achieved  by  flying  to  the  cued 
target  location  which  was  also  the  actual  target  location.  In  these  trials,  any  operator 
control  could  only  move  the  missile  path  away  from  the  cued  and  actual  target  location. 

3.  Extrapolating  NLOS  Target  Availability  to  FOG  Concepts 

To  produce  a  probability  of  target  availability  for  the  FOG  concepts,  the  technique 
described  in  the  preceding  section  of  combining  a  model-produced  projectile  flight  path 
and  sensor  orientation  with  SIMNET  target  paths  and  terrain  was  used.  In  Figure  2,  this 
process  starts  with  box  3.  Target  availability  results  for  the  NLOS  missile  could  not  be 
used  directly  for  the  FOG  concepts  due  to  differences  in  ranges  and  trajectories  among 
the  NLOS,  FOG-MP(81),  FOG-MP(120),  and  FOG-HM. 

A  deterministic  model  describing  the  behavior  of  both  the  FOG  projectile  and  its 
seeker  was  developed  for  each  FOG  concept.  Each  model  was  run  for  different  SIMNET 
trials  and  relevant  conditions  were  varied.  These  conditions  were  sensor  FOV,  sensor 
look-down  angle,  time  to  launch,  range  to  target,  and  the  maximum  detection  range  of  the 
sensor.  Each  model  run  produced  a  FOG  projectile  path  and  its  seeker  orientation.  These 
were  input  to  the  target  availability  program  together  with  appropriate  SIMNET  target 
paths  and  terrain.  The  target  availability  program  produced  a  series  of  curves  plotting  the 
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probability  of  a  target's  being  available  for  detection  against  each  of  the  independent 
variables. 

These  results  are  likely  to  underestimate  target  availability,  given  the  increase  in 
target  availability  of  the  NLOS  missile  between  manned  trials  and  trials  with  default 
missile  and  seeker  behavior.  The  availability  results  for  the  FOG  concepts  are  therefore 
scaled  to  reflect  the  potential  operator  contribution  to  target  availability  by  adding  0.18  to 
the  average  P(avail)  for  each  concept.  This  method  of  adjustment  was  chosen  for 
simplicity  and  illustration;  it  could  be  refined,  e.g.,  by  decomposing  the  trials  into  disjoint 
sets  of  similar  trials  and  calculating  a  separate  adjustment  for  each  set  of  trials. 

Figure  3  shows  two  curves  for  the  probability  of  target  availability  versus  delay  to 
launch  for  the  FOG-MP(120).  The  delay  to  launch  is  the  interval  between  initial  target 
acquisition  and  the  launch  of  the  mortar  round.  (The  time  of  initial  target  acquisition  is 
found  in  the  NLOS/SIMNET  trials.)  The  launch  time  was  varied  from  10  seconds  to  1 10 
seconds  after  initial  target  acquisition  in  20-second  increments.  In  Figure  3,  the  FOG- 
MP(120)  is  assumed  to  have  a  9°xl2°  sensor  FOV  and  a  1.7-km  maximum  sensor 
detection  range.  The  lower  curve  is  the  probability  of  target  availability  based  on  model 
results  only.  The  upper  dashed  curve  is  the  adjusted  probability  of  target  availability; 
0.18  has  been  added  to  reflect  the  potential  operator  contribution  to  availability. 

Availability  in  Figure  3  generally  decreases  as  the  delay  to  launch  increases 
because  targets  do  not  appear  in  the  sensor  FOV.  The  extra  dip  in  availability  at  the  50- 
and  70-second  launch  delays  is  due  to  the  shape  of  one  of  the  target  paths.  This  path  was 
a  semicircle  with  both  end  points  close  to  a  line  drawn  from  the  mortar  launch  site  to  the 
cued  target  location.  Hence,  the  targets  are  furthest  away  from  the  mortar  launch-cued 
location  line  for  the  intermediate  length  delays. 

D.  CREATION  OF  P(SSE) 

For  each  FOG  concept,  the  P(SSE)  is  the  product  of  the  adjusted  probability  of 
target  availability  and  other  conditional  probabilities  from  field  test  and  model  data,  as 
shown  in  box  4  of  Figure  2.  The  data  sources  for  the  three  remaining  conditional 
probabilities-detection  and  lock-on  given  availability,  hit  given  lock-on,  and  kill  given 
hit— are  described  below. 
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Figure  3.  Sample  Probabilities  of  Target  Availability  vs.  Delay  to  Launch  for  FOG-MP(120) 


1.  Target  Detection  and  Lock-On 

The  probability  that  the  gunner  detects  and  locks  on  to  the  target  is  based  on 
results  from  the  FOG-M  lOE.  One  purpose  of  the  lOE  was  to  determine  the  operator’s 
ability  to  detect  and  lock  onto  a  target,  given  a  target  presented  in  the  sensor  FOV.  The 
results  provide  estimates  of  the  average  conditional  probability  that  an  armor  target  is 
detected  and  locked  onto  and  the  average  conditional  probability  that  a  rotary  wing  target 
is  detected  and  locked  onto,  for  the  backgrounds  against  which  the  targets  were  presented 
and  their  state  (moving  or  stationary). 

2.  Target  Hit 

The  probability  that  an  armor  or  rotary  wing  target  was  hit  given  that  it  had  been 
locked  onto  was  also  evaluated  during  the  FOG-M  lOE.  These  values  are  used  for  the 
FOG  mortar  rounds  and  helicopter-launched  missile  under  the  assumption  that  they 
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depend  on  the  autotracker  software  that  maintains  lock  on  and  on  the  maneuver  capability 
of  the  projectile.  Preliminary  analysis  of  the  FOG  concepts  indicates  their  maneuver 
capability  would  not  be  a  limiting  factor  against  the  targets  used  in  this  analysis. 

3.  Target  Kill 

The  probability  that  an  armor  target  is  killed  given  that  it  has  been  hit  is  based  on 
data  from  runs  of  the  U.S.  Army  Ballistics  Research  Laboratory  Vulnerability  Analysis  of 
Surface  Targets  (VAST)  model.  The  data  of  interest  are  probabilities  of  kill  given  hit  for 
4-inch,  5-inch,  and  6-inch  shaped-charge  warheads  against  various  tanks.  Values  for 
specific  FOG  systems  are  produced  by  extrapolating  from  or  interpolating  between 
appropriate  probabilities. 

Data  for  the  probability  of  killing  a  rotary  wing  target  given  a  hit  were  not 
available,  but  this  was  assumed  to  be  0.90  for  the  FOG-HM. 

The  NLOS/SIMNET  tests  were  well  suited  to  the  analytic  method  just  described 
of  combining  a  modeled  system  with  SIMNET  data.  One  reason  is  the  close  similarity  of 
NLOS  to  the  FOG  weapon  concepts  of  interest.  Another  reason  is  that  the 
NLOS/SIMNET  test  consisted  of  many  small  vignettes  (trials)  and  was  not  concerned 
with  how  the  outcomes  of  the  vignettes  affected  a  larger  battle.  Nevertheless,  the  method 
of  developing  a  model  and  asking  how  the  system  (e.g.,  a  UAV)  or  procedure  (e.g.,  an 
engagement  rule)  represented  by  the  model  would  have  behaved  in  previous  SIMNET 
exercises  is  not  limited  to  the  NLOS/SIMNET  data.  This  technique  is  generalized  and 
explored  further  in  the  following  chapter. 
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IV.  METHODOLOGY:  THREE  TECHNIQUES 
FOR  ANALYZING  DIS  EXERCISES 

The  methodology  developed  for  the  FOG  weapon  analysis  has  been  expanded  and 
generalized,  producing  three  analytic  techniques  for  analyzing  DIS  exercises. 

One  technique  is  simply  extracting  and  combining  data  from  one  or  more  prior 
exercises  to  develop  measures  of  effectiveness  (MOEs).  In  addition  to  providing  insight 
into  the  original  exercise(s),  these  MOEs  might  be  of  general  tactical  or  methodological 
interest. 

The  second  technique  is  to  impose  a  model  on  a  DIS  exercise.  Data  for  systems 
produced  from  a  source  other  than  DIS  (e.g.,  a  deterministic  model)  can  be  applied  to  DIS 
data,  subject  to  certain  data  format  and  logical  constraints. 

In  the  third  technique,  a  logger  file  is  replayed  to  operator(s)  in  a  manned 
simulator  who  operate  the  simulated  system  on  the  previously  recorded  battlefield.  It  will 
appear  as  though  the  battle  is  occurring  at  that  instant,  subject  to  the  limitations  discussed 
below. 

In  addition  to  describing  the  three  techniques  and  providing  examples,  this  section 
also  explores  the  following  (perhaps  paradoxical)  proposition:  The  method  applied  to 
NLOS/SIMNET  data  and  its  extension  to  the  replaying  of  prior  SIMNET  or  DIS  exercises 
through  a  manned  simulator  is  both  severely  limited  and  extremely  rich  in  its  potential 
applicability. 

The  method  has  two  major  limitations.  First,  a  prior  exercise  is  history.  While 
the  logger  data  for  that  exercise  or  the  data  derived  from  that  tape  can  be  altered,  the 
extent  to  which  that  can  be  done  sensibly  is  very  limited.  In  the  main,  what  happened  in 
the  prior  exercise  may  be  reinterpreted  to  reflect  the  effect  of  a  new  system  but  cannot  be 
changed.  Hence,  the  tactics  on  both  sides  do  not  include  any  consideration  of  the  new 
system. 

As  another  example  of  this  limitation,  the  engagement  rules,  fly-out,  guidance, 
and  lethality  of  a  new  missile  can  be  modeled  and  applied  to  a  prior  exercise.  The  targets 
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can  be  engaged  by  the  model  just  as  if  the  new  missile  had  been  present  in  the  original 
exercise.  The  impact  point  of  the  missile  can  be  determined,  and  the  damage  to  the  target 
can  be  calculated  exactly  as  it  would  be  in  a  DIS  exercise.  But  the  target  does  not  die.  It 
continues  to  fight  until  the  point  in  the  battle  where  it  was  originally  killed  or  until  the 
conclusion  of  the  exercise.  One  might  take  the  analysis  a  step  further  by  looking  at  the 
kills  subsequently  caused  by  a  target  engaged  by  the  model  and  using  this  as  a  measure  of 
the  value  of  killing  that  target.  But  this  ignores  the  fact  that  killing  a  target  may  affect 
many  things  that  could  invalidate  this  measure  (e.g.,  another  enemy  vehicle  might  fill  the 
role  of  that  target  if  it  were  killed). 

As  a  consequence  of  this  limitation  the  proposed  methodology  cannot  be  used  to 
answer  questions  of  overall  force  effectiveness  associated  with  a  new  system.  Questions 
of  which  side  would  win  or  what  the  overall  force  exchange  ratio  would  be  and  similar 
force  effectiveness  questions  are  beyond  the  reach  of  the  basic  methodology,  although 
incorporating  military  judgments  into  the  replay  methodology  can  extend  its  reach.  On 
the  other  hand,  questions  concerned  with  opportunities  for  using  a  new  system  and  with 
the  expected  performance  of  the  system  if  used  in  various  situations  can  often  be 
addressed  by  the  methodology.  In  this  case,  the  analysis  of  the  new  system  is  being  done 
in  situations  that  resulted  from  human  behavior  and  decisions. 

The  second  major  limitation  is  that  a  model  of  a  system  (or  a  procedure)  is  limited 
in  the  extent  to  which  it  portrays  the  human  involvement  required  for  operation  of  that 
system.  The  prior  DIS  exercise  is  being  used  to  capture  the  human  element  in  the 
situations  to  which  the  model  will  be  applied.  But  the  model  itself  lacks  the  human 
dimension.  In  any  given  analysis  this  may  or  may  not  be  a  disqualifying  limitation. 

There  are  at  least  four  ways  of  partially  overcoming  this  limitation.  The  first  is 
illustrated  by  the  FOG  weapon  analysis.  It  proved  possible  to  estimate  the  difference 
between  a  deterministic  model  and  a  manned  simulation  in  the  NLOS  trials.  It  was  then 
assumed  that  this  difference  could  be  extrapolated  to  other  FOG  systems.  (It  remains  to 
be  seen  whether  this  trick  depends  on  a  set  of  fortuitous  circumstances  that  is  unlikely  to 
occur  very  often.)  The  second  approach  illustrated  by  the  FOG  analyses  is  to  use  field 
test  data  to  complement  the  analysis  of  prior  DIS  exercises.  A  third  approach  is  to  use 
military  expertise  to  help  determine  how  the  system  being  modeled  could  have  been  used 
in  prior  DIS  exercises  and  the  contribution  it  could  have  made  to  the  battle. 
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The  fourth  approach  is  to  replay  a  exercise  to  a  crew  in  a  manned  simulator.  For 
example,  the  characteristics  of  an  NLOS  simulator  could  be  modified  to  represent  a 
different  FOG  weapon  and  those  NLOS  trials  appropriate  to  the  new  system  could  be 
played  through  the  modified  simulator.  In  this  example,  the  crew  probably  could  not  tell 
that  the  exercise  was  not  occurring  for  the  first  time.  In  more  complex  exercises,  the 
crew  could  probably  be  fooled  for  awhile,  but  for  some  systems,  the  inability  to  affect  the 
battle  would  gradually  reveal  the  fact  that  the  exercise  was  being  replayed. 

In  spite  of  these  limitations,  the  FOG  weapon  analysis  suggests  that,  with  enough 
care  in  working  within  and  getting  around  the  limitations,  useful  results  can  be  obtained. 
The  methodology  used  for  that  study  both  allowed  the  NLOS/SIMNET  test  results  to  be 
extended  in  ways  not  planned  for  in  the  test,  e.g.,  the  comparison  of  manned  versus 
modeled  default  missile  and  seeker  operation,  and  also,  by  using  the  NLOS/SIMNET 
data,  was  an  order  of  magnitude  less  expensive  than  the  SIMNET  test  itself. 

A.  BASIC  PROCESSING 

Basic  processing  is  used  to  develop  results  from  one's  own  DIS  exercise  or  to 
develop  results  from  a  prior  exercise  to  be  applied  or  extrapolated  to  a  system  or  issue 
other  than  those  for  which  the  test  had  originally  been  performed. 

In  addition,  certain  general  tactical  knowledge  or  information  that  may  help  to 
calibrate  DIS  results  can  be  gained  from  previous  exercises.  Some  examples  include  the 
following: 

•  Time  and  range  required  to  detect  and  identify  targets 

•  Duration  of  LOS  among  vehicles  and  between  missiles  and  vehicles 

•  Movement  patterns  of  vehicles:  average  speeds,  paths,  and  areas  of  terrain 
used 

•  Frequency  of  overkill 

•  Probability  of  multiple  targets  available  for  engagement  given  one  target 
available 

•  Variation  in  operator  or  crew  effectiveness. 

Some  of  the  results  may  not  correspond  well  to  operational  results,  such  as  target 
detection  and  identification  and  terrain-related  measures,  since  they  depend  on  the  current 
computer  image  generators  and  terrain  database.  These  results  should  be  collected. 
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Comparisons  with  field  test  data  should  be  tracked  as  SIMNET  and  DIS  technology  is 
upgraded  and  displays  and  terrain  become  more  realistic. 

Conceptually,  the  data  reduction  approach  applied  by  IDA  to  several  SIMNET 
evaluations  consists  of  three  steps.  First,  a  subset  of  data  describing  all  vehicle  and 
missile  positions,  munition  impact  locations  (ground  or  vehicles),  and  vehicles  killed  is 
extracted  from  one  or  more  logger  files.  Second,  these  data  are  used  to  construct  a  state 
space  describing  the  x,y,z  positions  of  all  vehicles  and  missiles  at  every  second.  Third, 
the  MOEs  are  calculated  based  on  information  in  the  state  space  and  on  the  information 
describing  munition  impacts  and  vehicles  killed.  These  MOEs  might  be  killer/victim 
scoreboards,  target  availability  measures,  or  average  vehicle  speeds.  In  addition,  data 
describing  the  state  space  are  written.  If  multiple  exercises  are  used,  the  entire  process  is 
repeated  for  each  exercise. 

The  data  output  describing  the  state  space  can  be  inputs  to  a  graphing  program 
that  depicts  vehicle  and/or  missile  paths  over  time  or  second-by-second.  Plots  of 
vehicle/missile  positions  can  help  the  analyst  understand  the  geometry  of  the  battlefield 
engagements,  which  are  interesting  and  varied.  As  an  example,  for  the  NLOS/SIMNET 
evaluation,  the  position  of  the  targets  and  the  NLOS  missile  were  plotted  over  time  for 
individual  trials.  This  provided  insight  into  gunner  search  strategies  on  trials  where  the 
target  was  not  available  for  detection  at  the  cued  target  location.  The  gunner  might  veer 
off  to  the  left  or  right  and  search  an  adjacent  area,  or  the  gunner  might  fly  in  an  ever- 
widening  spiral  to  locate  the  target.  The  effectiveness  of  the  various  strategies  was  not 
studied,  but  the  data  to  support  that  work  are  available. 

If  it  is  desired  to  include  the  time  dimension  in  plots,  snapshot  plots  depicting 
vehicle  positions  at  a  specific  time  can  be  developed,  or  parts  of  an  exercise  can  be 
animated.  Both  methods  are  straightforward  once  the  state  space  of  the  vehicles  has  been 
created. 

B.  IMPOSING  A  NEW  SYSTEM  ON  A  DIS  EXERCISE 
1.  Method 

A  new  system  is  added  to  an  existing  DIS  exercise  by  providing  data  about  that 
system  as  an  additional  input  to  the  state  space  derived  from  a  logger  file  and  then 
calculating  MOEs  based  on  the  augmented  state  space. 
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More  specifically,  a  second-by-second  listing  of  the  system’s  x,y,z  positions  on 
the  battlefield  must  be  produced.  If  the  system  employs  a  sensor,  then  the  sensor 
orientation  and  FOV  at  each  second  must  also  be  produced.  These  positions  become 
inputs  to  the  creation  of  the  state  space  from  which  MOEs  are  developed. 

The  system’s  positions  can  be  produced  from  any  source  such  as  a  separate  model 
or  simulation.  For  example,  a  model  of  the  fly-out  of  a  new  missile  could  be  developed, 
producing  a  set  of  x,y,z  positions  describing  the  missile's  trajectory.  This  set  of  positions, 
along  with  DIS  data,  could  be  used  to  create  the  state  space  and  then  develop  MOEs. 

Because  of  the  inability,  noted  above,  to  change  the  vehicles'  behavior  in  a  prior 
exercise,  particularly  the  inability  to  kill  a  vehicle,  it  can  be  useful  to  subdivide  an 
exercise  into  smaller  segments  or  vignettes.  The  effect  of  the  new  system  can  be 
calculated  separately  for  each  vignette.  By  choosing  appropriately  sized  vignettes,  the 
problem  of  modifying  the  behavior  of  vehicles  that  would  have  been  affected  by  a  new 
system  is  minimized. 

2.  Applications 

•  Effect  of  Operator  on  System  Performance 

For  a  variety  of  systems  that  rely  heavily  on  operator  control,  the  effect  of  the  . 
operator  on  system  performance  can  be  measured  by  comparing  results  from  a  manned 
DIS  experiment  with  results  obtained  by  replacing  the  manned  system  (from  the  same 
experiment)  with  a  deterministic  version  of  the  system  produced  from  a  model.  An 
example  of  this,  determining  the  effect  of  human  operation  on  target  availability  for  the 
NLOS  missile,  was  described  in  the  previous  section.  This  type  of  comparison  can 
provide  some  understanding  of  the  contribution  of  the  operator  to  system  performance 
and  the  circumstances  under  which  operator-controlled  operation  is  most  or  least  useful. 
For  newer  systems,  such  as  unmanned  vehicles  (UVs),  either  air,  ground,  or  underwater, 
and  FOG  systems,  where  the  way  in  which  the  operator  employs  the-  system  may  not  be 
well  understood,  performing  a  DIS  experiment  with  an  operator,  comparing  the  results  to 
a  deterministic  model,  and  combining  that  comparison  with  a  detailed  review  of  operator 
techniques  may  help  refine  tactics  or  procedures  for  system  use. 
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•  Adding  New  Deterministic  Systems 

A  modeled  deterministic  system  can  be  added  to  a  prior  exercise  without 
replacing  an  existing  system.  In  this  case,  the  DIS  exercise  supplies  the  battlefield 
vehicles,  their  movement  and  interaction,  and  the  terrain.  The  analysis  generates  the 
model  of  the  new  system  and  rules  that  determine  its  use  in  the  DIS  exercise.  By 
computing  performance-related  MOEs  such  as  target  availability  and  vehicles  hit, 
competing  system  characteristics  can  be  evaluated,  e.g.,  what  range  is  most  appropriate; 
what  kind  of  search  algorithm  for  an  autonomous  round  is  most  effective;  for  an  airborne 
system,  what  flight  path  (trajectory,  speed,  and  altitude)  is  most  effective? 

This  approach  could  also  be  used  to  assess  target  acquisition  opportunities  for 
UAVs  and  the  value  of  the  intelligence  information  they  would  acquire.  Military 
personnel  would  review  the  scenarios  of  one  or  more  existing  DIS  exercises  (they  would 
not  know  the  actual  course  or  outcome  of  the  battle),  determine  UAV  launch  sites,  and 
preplan  UAV  flight  paths  over  the  battlefield.  A  model  would  generate  the  second-by- 
second  description  of  the  flight  paths  and  would  describe  the  sensor  FOV  and  orientation. 
The  state  space  would  be  constructed  using  the  UAV  path  and  sensor  characteristics  from 
the  model  and  the  paths  of  the  DIS  vehicles  from  the  prior  exercise. 

A  number  of  measures  for  target  acquisition  could  be  computed.  First,  target 
availability  for  the  UAV  could  be  calculated.  The  effect  of  different  UAV  characteristics 
on  target  availability  can  be  determined  by  varying  different  parameters  in  the  model, 
such  as  sensor  FOV  or  scan  pattern,  or  the  altitude,  speed,  or  range  of  the  UAV,  and  then 
repeating  the  process  of  running  the  model,  constructing  the  state  space,  and  computing 
target  availability.  After  an  estimate  of  target  availability  is  developed,  it  could  be 
combined  with  estimates  of  operator  detection  rates  for  available  targets  to  estimate  how 
many  targets  might  have  Been  acquired  by  an  operator.  Performing  availability 
calculations  for  the  ground  vehicles  in  the  prior  exercise(s)  as  well  as  for  the  UAVs 
would  determine  whether  targets  become  available  to  the  UAVs  before  they  are  available 
to  ground  vehicles,  and  if  so,  how  much  earlier  they  are  available. 

The  value  of  the  intelligence  information  acquired  by  the  UAVs  could  be  assessed 
in  the  following  ways.  The  percentage  of  vehicles  on  each  side  (red  or  blue)  available  to 
each  UAV  could  be  calculated,  in  order  to  understand  how  much  of  the  battlefield  was 
available  to  the  UAVs.  Information  on  targets  acquired  by  the  UAV  could  be  provided  to 
military  personnel  who  then  would  be  asked  to  identify  enemy  units,  predict  future  target 
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locations,  and  make  maneuver  decisions,  e.g.,  where  and  when  to  reinforce.  Their 
answers  could  be  compared  to  events  in  the  prior  exercise(s)  to  assess  how  the 
information  from  the  UAV  could  have  affected  the  battle. 

The  advantage  of  this  approach  is  that  it  allows  the  analyst  to  go  significantly 
beyond  the  limited  types  of  manned  simulators  currently  available.  In  addition,  many 
parameters  can  be  quickly  varied  in  a  controlled  way  that  makes  it  easier  to  determine  the 
effect  of  each  condition  on  system  performance.  For  example,  conditions  varied  in  the 
FOG  projectile  models  described  in  the  case  study  were  sensor  FOV  size,  sensor 
lookdown  angle,  time  to  launch,  range  to  target,  and  the  maximum  detection  range  of  the 
sensor.  Many  other  conditions  (target  location,  projectile  altitude  inflight,  etc.)  could 
have  been  varied  in  the  modeling.  In  addition,  the  SIMNET  data  could  have  been  altered. 
The  target  paths  could  have  been  moved  to  a  different  area  on  the  terrain  database,  the 
target  speed  could  have  been  increased  or  decreased,  or  the  number  of  vehicles  in  the 
target  array  could  have  been  increased  or  decreased.  This  approach  is  inexpensive,  and 
relatively  easy  and  fast  to  implement.  Performing  the  same  number  of  trials  in  a  DIS 
exercise  would  be  far  more  expensive  and  time-consuming. 

This  approach  is  most  readily  used  for  questions  that  involve  looking  at  the 
battlefield,  such  as  engagement  opportunities  for  weapon  systems  or  reconnaissance, 
surveillance,  and  target  acquisition  (RSTA)  opportunities  for  surveillance  systems. 

C.  REPLAYING  DIS  EXERCISES 

1.  Method 

Replaying  a  prior  DIS  exercise  to  an  operator  (or  crew)  seated  at  a  simulator  or  the 
Stealth  vehicle  allows  the  viewer  to  assume  the  position  of  any  simulator  (friend  or  foe) 
and  see  the  battle  exactly  as  the  simulator  occupant  did  or  to  move  freely  over  the 
battlefield  and  observe  the  action  from  any  position  and  altitude.  In  its  simplest  form,  the 
simulator  or  Stealth  vehicle  is  used  as  a  display  device.  The  operator  watches  the  tape  (as 
noted  earlier  he  may  or  may  not  perceive  that  the  battle  was  previously  recorded)  and 
performs  assigned  tasks.  Data  are  collected  on  his  performance  and  form  the  basis  for 
MOEs.  In  a  more  complex  form,  the  operator  could  view  a  prior  exercise  through  a 
simulator  and  he  would  be  able  to  use  the  simulator  as  a  weapon  system  on  the 
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battlefield.  The  simulator  could  represent  a  currently  simulated  vehicle  type,  or  it  could 
be  modified  to  represent  a  new  vehicle  type. 

This  technique  differs  from  the  previous  techniques  in  two  important  respects. 
First,  using  a  simulator  with  a  previously  recorded  DIS  logger  file  generates  a  new  logger 
file  that  adds  the  new  simulator  packets  to  all  the  previously  recorded  packets.  This  tape 
can  then  be  analyzed  and  MOEs  developed  using  the  state  space  technique  described 
earlier.  Second,  using  an  operator  or  crew  in  a  simulator  allows  human  behavior  to  enter 
directly  into  the  new  analysis.  The  operator’s  decisions  (where  and  when  to  move,  where 
and  when  to  fire)  can  be  recorded;  however,  this  method  is  still  limited  in  that  the  effect 
of  those  decisions  on  other  vehicles  must  be  inferred.  The  vehicles  in  the  previously 
recorded  exercise  cannot  be  affected  by  the  simulated  system  (except  analytically). 

For  simplicity,  the  applications  presented  below  describe  replaying  one  exercise  to 
an  operator  or  crew.  In  fact,  it  is  possible  and  potentially  more  interesting  to  replay  a 
number  of  exercises  to  the  operator  or  crew.  This  allows  operator  performance  and, 
where  applicable,  system  performance  to  be  tested  and  evaluated  in  a  variety  of  combat 
situations.  Different  exercises  may  occur  on  different  terrain  and  involve  different  kinds 
of  vehicles,  force  sizes,  and  missions. 

Theoretically,  this  approach  can  be  scaled  up  in  a  variety  of  ways.  A  command 
and  control  (C2)  radio  network  operated  by  other  people  can  be  added;  more  manned 
simulators  or  CGF  or  other  simulated  components  can  be  added.  Any  added  component 
that  generates  packets  will  have  those  packets  recorded  by  the  logger  file  program  and 
these  packets  will  also  become  part  of  a  new  logger  file.  However,  sensible  limits  on 
such  additions  have  not  been  developed.  Moreover,  the  larger  the  scope,  the  closer  the 
exercise  gets  to  a  full-scale  live  DIS  experiment.  At  some  point,  it  would  be  only 
marginally  more  time  consuming  and  expensive  to  perform  an  entirely  new  experiment 
that  would  yield  more  results. 

2.  Applications 

•  Target  Acquisition  for  UAVs 

The  ability  of  UAVs  to  acquire  targets  can  be  assessed,  varying  the  level  of  cueing 
information  they  receive  from  none  (the  UAV  is  simply  performing  routine 
reconnaissance)  to  specific,  information  about  numbers  of  enemy  units,  their  positions 
and  direction  of  movement,  and  the  time  of  the  sighting.  Similar  to  the  UAV  example  in 
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the  last  subsection,  military  personnel  could  review  the  scenario  of  a  prior  exercise  and 
determine  initial  locations  for  the  UAV.  Instead  of  preplanning  the  UAV  flight  path, 
however,  they  might  assign  a  RSTA  mission  to  the  UAV  simulator  operator.  He  would 
receive  a  cue  to  a  possible  area  of  enemy  movement  and  fly  the  UAV  to  that  area. 
Possible  MOEs  might  be  how  efficiently  does  the  UAV  reach  the  location;  how  many 
blue  and  red  vehicles  are  available  along  the  way  and  how  many  does  the  operator 
correctly  detect  and  identify;  how  does  a  delay  in  cueing  the  UAV  affect  the  availability 
of  targets;  how  does  the  time  to  the  cued  location  affect  target  availability;  and  how  do 
the  UAV  endurance,  range,  and  sensor  characteristics  affect  its  performance.  In  addition, 
operator  search  tactics  can  be  analyzed  and  compared. 

An  extension  of  this  experiment  would  be  to  assess  the  effectiveness  of  the  C2 
network  that  relays  the  target  acquisition  information  to  a  firing  system  and  the  effects  of 
attacking  the  targets  acquired  by  the  UAV. 

•  Assessment  of  NLOS/UAV  Pairing 

To  assess  the  potential  combat  contribution  of  the  NLOS  cued  by  a  UAV,  a  prior 
exercise  could  be  replayed  through  both  a  UAV  simulator  and  an  NLOS  simulator.  The 
UAV  operator  would  fly  over  the  battlefield  searching  for  targets  and  would  radio  target 
cues  to  the  NLOS  simulator  operator  either  directly  or  through  a  C2  network.  The  NLOS 
operator  would  fly  one  or  more  missiles  to  the  cued  target  location  and  engage  the 
target(s). 

Target  availability  MOEs  for  the  UAV  like  those  described  in  the  preceding 
example  could  be  assessed,  varying  the  scenario  (by  varying  the  prior  exercise  or  by  using 
different  parts  of  an  exercise)  and  varying  simulator  operators.  Target  availability  for  the 
NLOS  could  be  similarly  assessed.  In  addition,  the  effect  of  different  NLOS/UAV 
integration  concepts  on  NLOS  target  availability,  target  detection,  identification,  hits,  and 
kills  could  be  explored.  A  base  case  might  be  established  with  the  UAV  operator 
radioing  targets  directly  to  the  NLOS  operator.  Other  C2  configurations  might  be 
assessed  using  live  C2  networks,  if  C2  operator  workloads  are  realistic,  or  a  cue  delay 
might  represent  the  effect  of  the  target  cue  passing  through  a  C2  network. 

Insight  into  the  combat  contribution  of  the  NLOS  could  be  gained  by 
(analytically)  determining  the  number  of  targets  the  NLOS  would  have  hit  and  killed  and 
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then  determining  how  those  losses  (that  presumably  would  have  occurred  beyond  the 
direct-fire  battle)  would  have  altered  force  ratios  in  the  direct-fire  battle. 

This  example  can  be  generalized  to  evaluate  the  effect  of  various  C2 
configurations  on  other  kinds  of  weapon  systems. 

The  advantages  of  this  approach  are  that  it  incorporates  many  aspects  of  human 
capabilities  and  decision-making  into  the  analysis,  with  a  smaller  time  and  cost 
commitment  than  a  live  experiment.  In  addition,  a  number  of  SIMNET  and  DIS 
exercises  have  been  archived  and  the  logger  files  are  readily  available. 

D.  DISCUSSION 

Compared  to  computer  simulations,  analysis  based  on  prior  DIS  exercises 
incorporates  many  more  aspects  of  human  behavior  and  decision-making.  Moreover, 
such  analysis  can  be  done  quickly  and  at  a  low  cost,  using  readily  available  facilities  and 
software.  In  contrast,  planning  and  conducting  a  new  manned  DIS  exercise  is  expensive 
and  time  consuming.  Analysis  of  prior  exercises  should  be  considered  as  a  way  to  build 
on  the  investment  of  others,  including  routine  investments  in  replicated  training  exercises. 
Many  SIMNET  and  DIS  exercises  have  been  archived,  representing  hours  of  battle 
employing  a  large  variety  and  number  of  forces.  The  archived  logger  files  can  be , 
processed  using  commonly  available  hardware.  The  software  for  processing  can  be 
developed  in  any  language  and  is  fairly  straightforward. 

In  addition  to  evaluating  the  effect  of  new  systems  by  using  a  model  with  prior 
DIS  exercises  or  replaying  exercises  through  a  manned  simulator,  these  techniques  can  be 
used  to  expand  the  results  available  from  one’s  own  exercises.  Because  of  limits  on  time 
and  other  resources,  in  a  typical  DIS  exercise  a  relatively  small  number  of  parameters  are 
varied.  Additional  parameter  variations  can  be  accomplished  analytically  by  applying 
these  techniques  to  the  logged  data  from  the  exercise.  The  live  DIS  exercise  can  focus  on 
those  parameters  that  depend  most  heavily  on  human  involvement.  Thus,  a  larger  set  of 
results  is  obtained  by  combining  the  explicit  results  of  the  exercise  with  the  results 
obtained  by  using  these  techniques. 

The  two  major  limitations  of  the  methodology  have  been  discussed  extensively. 
First  is  the  fact  that  the  history  represented  by  a  prior  DIS  exercise  can  only  be  changed  in 
very  limited  ways.  Second,  the  analysis  of  a  new  system  using  a  prior  exercise  does  not 
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include  human  operation  of  the  new  system  if  it  is  represented  by  a  model.  Even  if  the 
new  system  is  represented  by  a  simulator  through  which  the  prior  exercise  is  played,  some 
important  aspects  of  human  operation  are  likely  to  be  lost. 

The  FOG  weapon  analysis  based  on  the  NLOS/SIMNET  test  shows  that,  in  at 
least  one  case,  it  was  possible  to  obtain  interesting  and  useful  results  by  analyzing  a 
previously  conducted  SIMNET  test.  Though  admittedly  a  special  case,  this  paper  has 
tried  to  show  that,  with  care  and  ingenuity,  a  surprisingly  large  range  of  questions  can  be 
addressed  with  a  similar  analytic  approach.  If  this  is  true,  then  the  importance  of  this 
approach  is  bound  to  increase  as  existing  manned  distributed  simulations  are  improved 
and  new  ones  are  developed. 

Barriers  to  wider  use  of  this  approach  include  the  lack  of  documentation  on 
exercise  post-processing  and  the  limited  number  of  carefully  archived  exercises  relative 
to  the  number  of  exercise  hours  played.  These  methods  would  be  more  easily 
implemented  if  portable,  simple  software  for  the  basic  tasks  of  reading  and  interpreting 
packets  and  performing  standard  calculations  (e.g.,  LOS  calculations  with  the  terrain 
database)  were  available.  Archiving  of  tapes  should  be  performed  more  consistently  and 
completely.  Currently,  SIMNET-D  sites  archive  most  logger  files  for  2  years.  SIMNET- 
T  policy  varies  by  site.  In  addition  to  the  logger  files,  other  kinds  of  materials  such  as  test 
plans  and  scenario  descriptions  must  also  be  archived. 

As  well  as  archiving  logger  data,  results  from  processing  those  tapes  should  also 
be  archived.  This  would  establish  a  database  of  results  from  a  variety  of  exercises.  These 
results  would  aid  understanding  of  how  DIS  simulators  behave  and  how  operators  behave 
in  DIS. 
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Appendix  A 
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C2 

CGF 

CP 

command  and  control 
computer-generated  forces 
command  post 

DARPA 

DIS 

Defense  Advanced  Research  Projects  Agency 

distributed  interactive  simulation 

HMMWV 

High  Mobility  Multipurpose  Wheeled  Vehicle 

IFF 

lOE 

identification  friend  or  foe 
initial  operational  evaluation 

FOG 

FOG-M 

FOV 

fiber  optic  guided 
fiber  optic  guided  missiles 

field  of  view 

km 

kilometer 

LOS 

line  of  sight 

MOE 

measure  of  effectiveness 

NLOS 

non-line  of  sight 

OPTEC 

Operational  Test  and  Evaluation  Command 
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P(SSE) 

probability  of  single  shot  engagement 

PDU 

protocol  data  unit 

RSTA 

reconnaissance,  surveillance,  and  target  acquisition 

SHORAD 

short-range  air  defense 

SIMNET 

simulation  network 

UAV 

unmanned  air  vehicle 

UGV 

unmanned  ground  vehicle 

uv 

unmanned  vehicle 

VAST 

Vulnerability  Analysis  of  Surface  Targets 
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